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The purpose of the Professional FiUisto present one or more possible 
solutions to a current problem in post -secondary education. The 
solutions that are described have been found to be effective on at least 
one campus. Topics and authors are approved by the PuJjlications 
Committee and the Executive CouncU of CSSHE. The Professional File 
ispubli^ed up to four times per year by CSSHE and is distributed free 
of charge to CSSHE members. Professional File 02 described a 
program review process as it was implemented at the University of 
Alberta, Proftssioital File « brieft, describes programs reviews at 
some other Canadian colleges and universities and provides an update 
of the University of Alberta's program review process after tenyears of 
implementation. This File has four co-authors, all of whom have been 
personally involved in the review process. Dr. Holdaway and Dr. 
Crawford were members of the committee which recommended that a 
systematic review process be established on campus and then prepared 
the preliminary policies and procedures. Dr. Harris served as 
Chairman of the President's Advisory Committee on Campus Reviews 
(PACCR) from 1980 to 1991. Prior to that appointment, he was 
Chairman of the Department of Chemistry. Dr. Crawford was 
Chairman of the President's Committee which examined the University's 
experience with the reviews after eight years and recommended 
procedures for a second cycle of reviews. Dr. MeeHson served as 
Vice-President (Academic) from 1984 to 1991 and was closely 
associated with the review process. 

Your comments and suggestions regarding this series are always 
welcome 

Norman Uhl and Gilles Jasmin, Co-editors 
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ABSTRACT 

Many universities in North America and elsewhere instituted various systems of reviews of their programs in the 1970s and 1980s. 
Program improvement was a common reason for such action, but providing information for planning and budget decisions was also 
important. Canadian universities use review formats which vary in purpose and scope. 

From 1980 to 1991, the University of Alberta undertook systematic reviews of aU its academic and support units: these were organized 
by the four-person President's Advisory Committee on Campus Reviews (PACCR). The main elements were preparation of a Self-Study 
Report, a site visit and preparation of a report by a four- or five-person Unit Review Committee, and preparation of a response by the 
unit. Various meetings were then held involving the President, PACCR, the unit head, and others. Long-term follow-up activities were 

caixied out 18 months later. ^ 
In 1988 the President established a committee to examine the University's 1980-1988 experience with program reviews and to recommend 
future action. This committee recommended continuation of the review process, but in a modified form. However, the University decided 
in 1991 not to initiate a second cycle of program reviews at this time. 
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SOMMAIRE 

Au couis des deux demi^res d^cennies, en Ara6rique commc ailleurs dans le monde, plusieurs universit€s ont entrepris 
des Evaluations destinies a am61iorer leurs programmes. Ccs Evaluations devaient 6galement pcrmettre d'assembler Tinformation 
nEcessaiie a une meilleure planification organisationnelle et budgEtaire. Les Evaluations rEalisEes par les universitEs canadiennes 
sent trds diverses, autant dans leurs mEthodes que dans leurs objectifs. 

Entre 1980 et 1991^ TUniversitE de I'Aiberta a rEalisE des Evaluations systEmatiques de chacune des unitEs 
d'enseignement, de recherche et d'administration de sa structure organisationnelle. Un comitE consultatif de quatre pcisonnes, 
relevant du bureau du Recteur, a Ete chargE de la coordination de I'opEration {President's Advisory Committee on Campus Reviews 
— PACCR). Toutes ces Evaluations avaient en commun les ElEments suivants: une auto-Evaluation par chaque unitE, un examcn 
sur place et un rapport d'Evaluation prEparE par un comitE indEpendant composE de 4 ou 5 pcrsonnes, et enfin, un contre-rapport 
prEparE par I'unitE. Au terme de cette phase de I'opEration, diverses rEunions ont EtE tenues auxquelles ont paiticlpE le Recteur, 
le comitE consultatif, les chefs d'unitEs et divers autres pcrsonnes intEressEes. Un suivi a EtE fait dix-huit mois plus tard. 

En 1988, le Recteur a chargE un groupe de travail de faire le bilan des Evaluations rEalisEes depuis 1980 et de proposer 
des orientations pour Tavenir. Ce groupe de travail a recommandE que les Evaluations se poursuivent, mais dans une forme 
modifiEc. En 1991, TUniversitE a dEddE de ne pas entreprendre un second cycle d'Evaluations, du raoins dans I'immEdiat. 



Reviews of academic and support units in 
postsecondary institutions hav^ become 
increasingly common in the United States, 
Canada, and certain other countries since about 
1970. In 1980, the University of Alberta 
initiated a system of reviews of its 80 academic 
and 50 support units; the primary purpose was 
improvement of quality of operations. These 
reviews, organized by the President's Advisory 
Committee on Campus Reviews (PACCR), 
involved preparation of a Self -Study Report by 
each unit, an assessment by a Unit Review 
Conmiittee, and a response by each unit. The 
procedures were little altered during the 1980- 
1991 period. Determination of actions based 
upon reconmiendations in the reports was the 
responsibility of the department, the 
dean/director, or the President and appropriate 
Vice-President. In 1988, a committee 
established to examine the PACCR process 
reconmiended establishment of a second cycle of 
reviews, with some modifications in procedures. 
However, after completing reviews of virtually 
every academic and support unit on campus from 
1980 to 1991, the University of Alberta decided 
not to initiate a second cycle of program reviews 
at this time. 

This paper outlines relevant literature, the 
review practices used in some Canadian 
universities, the review process used at the 
University of Alberta, the issues involved, the 
findings and recommendations of the examining 
committee, and suggestions for postsecondary 



institutions. 



RELEVANT LITERATURE AND 
CONCEPTUAL FRAMEWORK 

This brief review of relevant literature is 
based upon that provided in the CSSHE 
Professional File # 2 by Harris and HolHaway 
(1987). Because much of the content provided 
below appears in many different publications, not 
all of the sources listed in the References are 
included in the text of this paper. 

The various purposes of program review at 
universities have been identified as follows: (a) 
to improve programs; (b) to inform decision- 
makers about strengths and weaknesses of 
programs; (c) to help institutions make decisions 
about installation, continuation, modification, 
expansion, or termination of programs; (d) to 
provide information for planning; (e) to 
determine the relative status of programs with 
respect to either specified standards or other 
programs, (f) to help an institution make 
decisions concerning expenditure reductions and 
increases in efficiency; and (g) to demonstrate 
accountability. 

After noting that only three Australian 
universities were conducting reviews routinely, 
Taylor et al. (1988) listed these benefits from 
regular reviews: 
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1. they will be more valuable and 
effective than if they are in response to 
some crisis, when individuals may feel 
under threat; 

2. they provide an opportunity for self- 
study and stocktaking at regular 
intervals; 

3. the collection of the performance 
data, which are to form pari (but only a 
partj of the raw material of the reviews, 
IS simpler if it is carried out as a regular 
routine, and in addition this allows 
trends to be appreciated, (pp. 4-5) 

Further, Wilson (1987) noted that although 
"many features of our society work against these 
evaluations ... the importance of the task and 
the very subtlety and complexity of its goals 
make assessment compelling for those who 
commit themselves to higher education" (p. 238). 
However, Skolnik (1989) offered an important 
caution: "The central focus in most program 
review [in higher education] is quality. Yet the 
literature on quality emphasizes that quality 
remains an elusive concept. . . ." (p. 627). As 
program reviews became common in North 
America, their scope increased, their procedures 
became more standardized, and they have been 
increasingly demanding of resources (Harpel, 
1986). 

In this paper, "program" refers to a 
department or cluster of departments rather than 
to academic programs. "Program review," then, 
is a comprehensive description and assessment of 
the history, current performance, and future plans 
and prospects for a department. Different 
universities use different review procedures. 
Some concentrate upon units, some upon 
programs, and others upon graduate programs of 
individual departments. Harpef (1986) stated 
that "there is no single correct method for 
performing program review" (p. 2). 

Program reviews have also received 
considerable attention in Australia. As noted by 
Taylor et al. (1988): 

The process of national reviews of 
disciplines has already begun with Law, 
Medicine and Engineering, and should 
undoubtedly be continued, but given the 
extent of the subjects to be reviewed and 



the resources which might be devoted 
to such an exercise, it is unlikely that 
any one area will be reviewed more 
often than once in every fifteen or 
twenty years. On the other hand it 
ought to be possible for institutions to 
conduct their own internal reviews more 
frequently, so that every academic unit 
could be reviewed at least every ten 
years, (p. 4) 

Conrad and Wilson (1985) considered that 
"program reviews at most universities draw 
heavily on one or more of several models: goal- 
based, responsive, decision-making, or 
connoisseurship" (p. iv). They saw the goal- 
based model (Tyler, 1949) as being the most 
influential. The responsive model (Scriven, 
1972) addresses program issues, activities, and 
concerns, regardless of goals. Linking 
evaluations with decision making is seen to be 
the main contribution of the decision -making 
model (e.g., Stufflebeam et al., 1971). The 
connoisseurship model (Eisner, 1975) relies 
heavily on expert opinion, valued because of 
superior knowledge and expertise and a common 
value system. 

Several writers (e.g.. Craven, 1980, p. 434) 
have emphasized the need to evaluate the review 
processes used at universities, i.e., to "review the 
reviews." Wilson (1987) concluded that 
"although . . . [department] reviews have become 
more common over the last twenty years, they 
have not been much thought about or analyzed 
until quite recently," and that "we are, in fact, 
quite ignorant of the actual effects induced by 
the department review process" (p. 238). 

With respect to the benefits and costs of 
reviews, various authors have proposed similar 
lists. The most commonly mentioned benefits 
are as follows: (a) matters which are frequently 
ignored have to be examined; (b) planning is 
facilitated; (c) a greater sense of "conmiunity" 
may be generated in a unit; (d) those likely to be 
affected by changes are involved in the 
discussions and are therefore more likely to be 
knowledgeable and receptive; (e) the institutional 
image can be improved; and (f) qualitative 
information is provided to balance typical 
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institutional data. The major costs are identified 
as either direct — salaries, honoraria, travel and 
subsistence, supplies, and photocopying — or 
indirect — increased workload for members of the 
unit and for senior administrators. To improve 
the ratio of benefits to costs, these suggestions 
have been made: (a) concentrate upon major 
objectives; (b) collect only the most relevant 
data; (c) use a credible, acceptable, continuing 
review system; (d) use effective and timely 
communications; and (e) concentrate more upon 
evaluation than data collection. 

Following their assessment of universities' 
experiences, Conrad and Wilson (1985) 
concluded that opinions are divided over the 
impact of reviews — some see them as beneficial, 
some as making no difference, and some as 
having negative impacts. Taylor et al. (1988) 
drew these conclusions: 

Finally, it must be recognized that there 
is no point in conducing reviews or 
evaluations unless their outcomes have 
some effect, and mfluence the future 
operations of an institution. 

... there must also exist mechanisms 
whereby the results of an evaluation ... 
can be regularly incorporated in the 
academic plannmg ana management 

grocesses or institutions. This is one of 
le reasons for the recommendation that 
the evaluations and reviews should be 
conducted as a matter of routine rather 
than as something exceptional, (p. 5) 

Conrad and Wilson (1985) recommended 
that "given the plethora of program reviews at all 
levels of higher education, the need to study the 
effects of such reviews more systematically is 
urgent" (p. vii). 

Another aspect relevant to reviews involves 
the increasingly common use of performance 
indicators in higher education. However, as 
Taylor et al. (1988) noted, such "indicators are 
an aid to good judgement and not a substitute for 
it" (p. 1). They continued: 

The need for institutions to be 
accountable and transparent can best be 
met not by the collection of indicators as 
such, but by their use in the context of 
a process of expert review. In that 
process, performance indicators form 



CSSHE PROFESSIONAL FILE, Number 9 

part of the necessary raw material of 
evaluation and assessment . These 
reviews give rise to judgements and 
decisions which, though they may make 
use of information supplied by the 
collection of data such as performance 
indicators, are, in the end, made by 
individuals or groups of people, (pp. 
1-2) 

The majoi elements of program reviews at 
universities are represented in Figure 1. This 
conceptual framework can readily be modified 
according to different institutional or personal 
perspectives. 

PROCEDURES USED IN 
CANADIAN UNIVERSITIES 

In order to obtain an overview of the 
procedures used in other Canadian universities, 
the Vice-Presidents (Academic) of 17 
universities across Canada were asked in 
September 1990 to provide information about 
their program reviews. Replies were obtained 
from all Vice-Presidents. Considerable diversity 
was apparent in the approaches used; these were 
analyzed using the dimensions identified below. 

Of the 17 universities, nine used a 
structured, cyclical approach, five a structured, 
non-cyclical approach, and two a non- 
structured, non-cyclical approach. The identified 
frequencies of cyclical review for each unit or 
program ranged from 5 to 10 years. The most 
common approach was exemplified by Manitoba 
where each faculty, school, and department was 
reviewed at least once every seven years using 
standardized procedures for both the self-study 
and external review under the supervision of an 
Academic Review Committee. Also, aU Ontario 
graduate programs were subject to review by the 
Ontario Council on Graduate Studies. 

Concerning the review focus, 12 universities 
identified academic units/programs, while 3 of 
these also identified support units. The graduate 
and undergraduate programs were specifically 
identified as foci by 3 and 5 universities. 
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respectively. At Dalhousie, for more than 15 
years, departmental graduate programs have been 
evaluated at five to seven-year intervals, while 
undergraduate reviews "have often been held in 
conjunction with the appointment of 
departmental chairpersons." 

Eight of the described procedures had been 
approved by the university senate whereas two 
had received final approval by Vice-Presidents, 
With respect to the group or person in charge, 
considerable variation was obvious. In two 
universities, the senate was identified, in five 
others a senate committee, while in two more a 
non- senate conmiittee was in charge. Vice- 
presidents organized the reviews on two 
campuses. At St, Francis Xavier University, the 
Conmiittee on Academic Reviews consisted of 
the Academic Vice-President, Vice-President 
Administration, Dean of Arts, Dean of Science, 
and five tenured Faculty members (three-year 
terms) elected by a University Senate; the 
Presidenf appoints the Chair for a three-year 
term. 

The most commonly identified objective of 
the review process was improving academic 
performance (mentioned by 8 of the 16 
universities); also identified were obtaining 
information for planning, budgeting, and setting 
priorities (5), ascertaining the extent to which 
objectives are met (4), ascertaining the extent to 
which community needs are met (4), assessing 
the calibre of a unit or program (2), developing 
departments (1), and assessing the suitability of 
objectives (1). 

TTie general procedure employed in the 
responding Canadian universities included a self- 
study report, a review conmiittee report 
following a site visit, and a unit response. Eight 
of the ten uinversities which provided 
information about their self-study reports 
specified details about what was to be included; 
one university had precise requirements, while 
another had broad, general guidelines. Only 
McGill University seemed to use a policy of 
public disclosure of the results of reviews, using 



The McGill Reporter for this purpose. 

The final aspect involves the composition of 
unit review committees. Use of two external 
reviewers was the most common practice, with 
or without varying numbers of internal 
reviewers. Input was usually obtained from the 
unit about suitability of the external reviewers. 
General guidelines were commonly provided for 
the unit review committees with respect to their 
activities while on campus and the nature of their 
reports. Disnosition of reports and appropriate 
follow-up activities were usually identified in 
the descriptions of procedures. 

UNIVERSITY OF ALBERTA 
PROCEDURES 

Because detailed description of the program 
review procedures used at the University of 
Alberta was published earlier (Harris, and 
Holdaway, 1987), the information provided 
beJow briefly summarizes that article and 
supplements it with new content. 

Background 

In December 1990 the University of Alberta 
had an emoJlment of 21,871 full-time 
undergraduate students in 17 faculties and one 
school, 2,807 full-time graduate students in 78 
departments, and 4,373 part-time students. The 
program review process had its origin in a 
Universities Priorities Committee 
recommendation in 1978 — when full-time 
undergraduate and graduate enrollments were 
considerably smaller, being 16,757 and 2,007 — 
that reviews of all programs and units be 
initiated. After considerable discussion and 
assessment of procedures used at other 
institutions, the University endorsed these 
principles and procedures: 

1. The goal should be production of 
information that would serve as a reliable basis 
for planning and program improvement. 
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2. Administration of reviews should be 
readily understood effective, efficient, 
responsive, flexible, credible, and realistic. 

3. A small President's Advisory Committee 
on Academic Reviews (PACAR) should be 
established to organize the reviews. 

4. PACAR will advise the President on the 
sequence of units to be reviewed and the 
composition of Unit Review Committees 
(URCs), wiU advise units and URCs about 
procedures, and will transmit re;K)rts to the 
President. 

5. Each unit will prepare a Self-Study 
Report. 

6. Tne URC for a unit will consist of two 
members of the same discipline from outside the 
University, one from a related discipline within 
the University, and one from the University-at- 
large. 

7. The reviews wiU be conducted over 
several years. 

8. Each URC should consult actively with 
students, staff, recent graduates, and employers 
of graduates if appropriate. 

9. The Vice-President (Academic) will , be 
responsible for monitoring action taken on 
recommendations. 

Soon after PACAR began operating, the 
University also decided to review all 
administrative/support units (hereinafter called 
^'support units'"), and the committee's name was 
changed to the President's Advisory Committee 
on Campus Reviews (PACCR). Four members 
of PACCR were appointed in 1980. For most of 
the 11-year period, it consisted of a Chairman (a 
professor emeritus), two professors emeriti, and 
the Director of Technical Services. The 
following principles guided PACCR's operations: 
(a) the reviews are to promote forward-looking 
change and high-quality scholarship; 



(b) multiple benefits will be expected; (c) 
considerable flexibility and minimum direction 
wiU be emphasized; (d) a limited number of 
reviews will be conducted initially; (e) PACCR 
and accreditation reviews will coincide whenever 
possible; and (f) sensationalism will be avoided 
and confidentiality respected. PACCR decided 
that a balance was required between numerical 
data, descriptive information, and evaluation. 
For several years the Office of Institutional 
Research and Planning provided standard sets of 
data for many units to use in their self-study 
activities. 

Overall Process 

In 1980-81, a cautious start was made with 
only five units so that workable procedures could 
be developed. After the first year about 15 units 
were reviewed annually. The units then 
completed a Self-Study Report (SSR) which 
typically took 6 to 12 months. SSRs were 
expected to address unit history, current status, 
key issues, strengths, areas for improvement, 
strategies for and commitments to changes, and 
roles of individuals and groups. 

The Unit Review Committees (URCs) 
sometimes included one member of a relevant 
profession. The units provided nominations for 
the external and internal-related members of the 
URCs. PACCR then discussed with the heads of 
the units both these nominations and their own 
nomination for the member-at-large. Until 
1989, the list of nominees was submitted to the 
President for approval. The 177 Canadian 
reviewers which have been used came from 26 
different universities and 13 outside agencies in 
9 provinces. The University of British Columbia 
and the University of Toronto provided the most 
reviewers. Hie 66 United States reviewers came 
from 37 universities and 3 outside agencies in 23 
states. 

The SSR and relevant background 
information were forwarded to each URC 
member. URCs were asked to address strengths 
and weaknesses of the department's teaching and 
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research programs, staff and staff workloads, 
adequacy of the resources provided, effectiveness 
of the organization of the department, relations 
of the department with others, its plans for the 
future, opportunities for improving the 
department's programs, its relations with internal 
units and external organizations, and possibilities 
for better utilization of resources available. 

Each URC sent its report to PACCR; it was 
then forwarded to the President and to the unit 
which prepared a Unit Response. PACCR 
members then met with the unit head to discuss 
the main features of the three reports and the 
conclusions. Following this meeting, PACCR 
also prepared a confidential summary for the 
President who discussed this summary with the 
PACCR members; the appropriate Vice- 
President joined this dis'»;ussion in 1989. 
Separate discussions of the SSR, the URC 
Repc;rt, and the Unit Response then occurred 
involving the President, the unit head, the vice-- 
president/dean/director to whom the unit head 
reported, and, a few months later, the Planning 
and Priorities Committee. Administrators were 
free to take action, at any time on the 
recommendations. Appropriate Vice-Presidents 
were responsible for follow-up activities. When 
Dr. Horowitz was President, he administered the 
budgeted funds available for implementing some 
of the recommendations made by the URCs, 
after consultation with the Vice-Presidents. Dr. 
Davenport, the current President, used a different 
approach: he received recommendations from 
the Vice-Presidents and reported them to the 
Planning and Priorities Committee. 

Two modifications were made in 1985. 
First, after meeting with the President to discuss 
reports, PACCR wrote a confidential letter to 
URC members to report the results of the review 
to date. Second, about 18 months later PACCR 
brought to the attention of the Vice-President 
(Academic) those short and long-term matters 
whose disposition needed reviewing. 

The PACCR Chairman, who held a 0.5 PTE 
appointment, made initial contact with the units. 



monitored programs, established the URCs, and 
generally directed the PACCR Office. He was 
assisted by a full-time Coordinator who 
interacted continually with the imits being 
reviewed. When PACCR members were not 
readily available, the Coordinator had authority 
to act on behalf of PACCR. 

No PACCR reviews were conducted in 
1980-1991 of departmentalized faculties 
following reviews of their individual 
departments. The Faculty of Graduate Studies 
and Research, which has responsibility for 
overall coordination of graduate programs, 
underwent a regular PACCR review in 1986-87. 
The only major deviation from established 
procedures was that the URC Report was 
distributed to all departments having graduate 
programs; this was deemed desirable because 
matters identified in the Report affected most 
academic departments. 

Information about the PACCR process was 
widely distributed on campus, and several related 
articles were published in refereed journals 
throughout the 1980s. Also, at the request of the 
Chair of the Board of Governors, the Tenth 
Annual Report of PACCR was distributed to all 
members of the provincial legislature, all 
members of parliament from northern Alberta, 
the mayor and all aldermen of the City of 
Edmonton, and several prominent local business 
leaders. 

Issues 

As could be expected, the comprehensive 
review procedure generated several issues. The 
following issues which were addressed in the 
Professional File # 2 (1987) are not included in 
this paper: membership on URCs, nature of 
URC reports, time problems, review of support 
units, rank ordering of units, involvement of 
students in reviews, teaching loads, the teaching- 
research relation, unit renewal ar.d resources, and 
reviews of professional depaitments. In this 
section, 13 other issues are presented 
alphabetically. Most of them were identified by 
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some of the URCs. 

Administration of academic units. URCs 
frequently commented that the structure and 
administration of units were often given 
insufficient attention because they were viewed 
as of lesser importance than academic activities. 
However, inadequate administration can 
adversely affect the experience of students. A 
common criticism by URCs was 
overinvolvement of staff in committees. Also, 
most URCs did not approve of assigning the 
program counselling of students to administrative 
officers. 

Aging equipment. Scientific and technical 
equipment must be kept current and in good 
working condition if students are to receive 
relevant education and if universities are to be at 
the forefront of research and advancement of 
knowledge. This is expensive, but, if units are 
to be effective, these actions must be taken. 

Communications. Many URCs commented 
on the need for better external and internal 
communications for both academic and support 
units. Almost without exception, the URCs 
pointed out the need for support units to better 
acquaint the campus with the services they have 
to offer. 

Consequences and contributions. In its 

Seventh Annual Report (1987), PACCR stated 
that "it is clear to the members oi PACCR that 
many important changes have been brought 
about as a result of the review process, 
particularly when a unit wants the effort to be 
productive" (p. 2). With respect to how units 
assessed the success of the review process, 
PACCR observed that in the early 1980s the 
obtaining of new positions was sometimes a 
criterion, whereas in the later years units were 
more aware of the value of reviews in the long- 
range process of program improvement (p. 2). 

Three specific examples of improvement 
were provided in PACCR's Seventh Annual 
Report. First, the Registrar considered that the 



URC analysis of problems about registration 
procedures encouraged his office to adopt a 
system of telephone registration more quickly 
than might otherwise have occurred. Second, the 
Department of English reported in its Unit 
Response that the extensive involvement of 
virtually all staff members in the response led to 
more careful attention to problems and to 
improved morale. Third, the Faculty of 
Extension was complimentary about the 
contribution of the self-study process toward 
increasing awareness of role, challenges, 
strengths, limitations, and issues. Also, URC 
Reports provided stimuli for reflection and action 
in their recommendations, criticisms, and 
suggestions. PACCR also made this candid 

assessment in its Seventh Annual Report (1987): 

We do not wish to pretend, however, 
that the results of reviews have been 
uniformly satisfactory. Much depends 
on the attitudes of the people m 
departments. When there is no will to 
bnng about important changes, then 
little will happen. When there is such 
a will, particularly within the leadership 
of a unit, then much can and does 
happen, (p. 11) 

Development of academic departments. 

Another matter mentioned in PACCR's Eighth 
Annual Report involved comments by some 
URCs about the life history of academic 
departments. The life cycle mcludes "initiation, 
a period of struggle to become established, a 
period of growth, and plateau period . . . [and] a 
penod of decline through loss of staff or morale 
and displacement of goals" (p. 12). However, 
PACCR concluded that "careful monitoring and 
early detection of symptoms" can and should 
lead to appropriate action which can generate 
renewal. 

Graduate programs. The University of 
Alberta included both undergraduate and 
graduate programs in the reviews of academic 
departments. While recognizing that graduate 
work is a highly individual undertaking and that 
departments must be given the widest degree of 
flexibility to ^csign programs for their students, 
PACCR urged that special attention should be 
paid by both the department and the URC to 
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admission procedures, to the monitoring of 
progress, and to supervision of graduate students. 

Reviewers indicated thnt it often takes too 
long for students to meet graduate degree 
requirements. The graduate thesis should be 
looked upon as a study introducing the candidate 
to the field of research, rather than as a definitive 
work on a given subject. 

Handling major changes. Modem 
universities place increasing emphasis on 
research and scholarship. For some departments, 
this means facing the challenge of moving away 
from earlier emphases upon teaching and service. 
Besides modifying aims and programs, staff have 
to be reoriented. URCs have pointed out that 
such changes are difficult and that they require 
careful administration. They also noted the need 
to treat fairly those faculty members whose 
careers have developed under a different set of 
expectedons. 

Mission statements. Although PACCR was 
decidedly cool to development of mission 
statements, believing that the missions of units 
were well known, URCs frequently 
recommended that such statements be prepared. 
In its Eighth Annual Report (1988), PACCR 
recognized that mission statements sometimes 
can help units focus their priorities where 
resources are limited; 

Replacement of retiring staff members. 

URCs often remarked that a significant number 
of staff members in a given department will be 
retiring in the next few years, and increasingly 
they have addressed tht question of how they 
should be replaced. A frequent recommendation 
was that a department not replace a retiring staff 
member with a person in the same area of 
specialization, so that the department could move 
into new areas, or that relatively neglected areas 
within the department should be strengthened. 
PACCR recommended that URC comments 
concerning future appointments or directions be 
given to appointment committees. 



Reviews and accreditation. In some 
professional faculties, concern existed about the 
overlap between PACCR and accreditation 
reviews, with the suggestion being made that the 
PACCR process be modified for professional 
departments. However, the purposes of the two 
types of reviews are different; accreditation 
relates to whether a unit is meeting the standards 
required for continuation, whereas the PACCR 
approach is directed towards improvement and 
planning. As part of the process of evaluating 
PACCR procedures, a faculty member with both 
PACCR and accreditation experience made 
these conmients which were reproduced, with the 
consent of the faculty member, in PACCR's 
Ninth Annual Report (1989): 

1. The PACCR review, in mv 
estimation, is by far the most detailed 
and comprehensive of all the reviev/ 
processes I have been involved in. . . . 

2. It forces a department chairman and 
his colleagues to examine themselves in 
their current context in comparison with 
other departments across the country 
and in related countries such as the 
U.K. and the U.S.A. Because of 
PACCR's firm (and correct) refiisal to 
supply detailed forms (which often pose 
irrelevant, inapplicable, or unanswerable 
questions) the tree-form presentation is 
left largely to the initiative of the 
department and its staff. This is a 
major benefit. One has to stand back 
and assess current and future issues in 
one's discipline as objectively as one 
can knowing that one will have external 
reviewers of such calibre that any 
dishonesty will be revealed 
immediately. 

3. In the PACCR system the reviewers 
are free to interview people not listed as 
well as to reinterview. (This is also 
theoreticaHy true of other surveys but 
time is so limited that, in practice, this 
is usually impossible logistically.) 

4. A key to the success of the PACCR 
review is the careful and aporopriate 
selection of reviewers. By picKing men 
or women who are eminent, respected 
and fair and who mostly wilT have 
encountered similar problems in the 
same discipline one ensures 
understanding of local and individual 
difficulties of funding, shortages of 
staff, and the rest, wmch may escape 
the attention of others. . . . 

5. Another factor adding to the quality 
of the PACCR reviews is that most 
disciplines in Canada are reasonably 
small communities and we meet each 
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other on a regular basis* so the reviewer 
must be responsible and be able to look 
his colleagues in the eye without, at the 
same time, playing favorites or douig a 
whitewash job. 

Reviews of non-standard units. Institutes 
and centres, which are usually attached to 
departments or faculties but which sometimes are 
free-sfanding, may be ^'uhierable in times of 
financial restriction. Regardless of whether or 
not they have advocates, non-standard units 
should neither benefit nor suffer 
disproportionately because of their special status. 

Teaching service function of departments. 

Departments in the Faculties of Arts and Science 
have long ago accepted the principle of service 
to other faculties. That, however, is not always 
the case in professional faculties. URCs have on 
a number of occasions commented on the 
difficulties experienced by some departments 
when they need seivice courses. Some 
departments have felt compelled to appoint staff 
members to teach such courses. URCs have 
generally disapproved of such a practice. 

Use of resources. URCs have commented 
that, in an environment of decreasing overall 
resources, the University and its units need to 
examine their operations carefully to ensure that 
resources are employed most effectively. They 
pointed to three areas in particular: 
(a) increasing cost recovery from trusts and 
grants; (b) restructuring of curriculum to ensure 
that it is relevant and current; and (c) assigning 
of support staff to assist a whole group of 
researchers rather than being assigned 
exclusively to one or two person^. 

RECOMMENDATIONS FOR 
FUTURE REVIEWS 

PACCR presented a set of recommendations 
in its Seventh Annual Report (1987) concerning 
how program reviews should be conducted once 
the first round was completed. After reflecting 



upon its procedures and alternatives, PACCR 
provided these ten recommendations: 

1. That the PACCR process be repeated and 
built into the administrative structure of the 
University. 

2. That the central role of the President be 
recognized and maintained. 

3. That the Advisory Committee for 
steering the review process be made up of senior 
members of the University community who have 
wide experience and sound judgment, but do not 
hold administrative positions. 

4. That the deans for academic units and 
associate vice-presidents for 
administrative/support units have the main 
responsibility for monitoring progress towards 
achieving agreed-upon recommendations arising 
from reviews, with the Vice-President 
(Academic) or Vice-President (Administration) 
retaining overall monitoring responsibilities. 

5. That the President have a reasonable sum 
of money each year to be used to respond to the 
findings of reviews. 

6. That program improvement continue to 
be the major objective of reviews. 

7. That the reviews be carried out as part of 
a process in which a unit examines its current 
strengths and limitations and with a strong 
forward-looking emphasis. 

8. That the reports of Unit Review 
Committees continue to be treated with 
considerable confidentiality. 

9. That single review teams, typically 
involving two external and two internal 
reviewers, continue to be the norm. 

10. That practices and procedures currently 
being used be carefully reexamined and adapted 
for the longer-term future. 
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EVALUATION OF THE PACCR 
PROCESS 

By December 1988, when over 80% of all 
academic and support units had been reviewed, 
President Myer Horowitz, as Chairman of the 
General Faculties Council, asked that a small 
committee examine the future role of PACCR. 
(The General Faculties Council is equivalent to 
a Senate in many other universities.) 

This Examining Committee, which consisted 
of an Associate Dean of Science as Chairman, 
the Dean of Rehabilitation Medicine, and the 
Registrar, conducted extensive interviews and 
obtained 90 written responses from University 
staff who had served on URCs. Three general 
questions were addressed; the statements of these 
questions and responses to them are provided 
below. 

Question 1: "What is your perception of the 
benefits derived from the PACCR process, 
and have these benefits outweighed the direct 
and indirect costs?" 

Cost-benefit relationships. While 24 
respondents assessed that the benefits resulting 
from the PACCR process outweighed the costs, 
5 provided a contrary opinion, and 11 were 
either undecided or could not make a judgment. 
Three said that the benefits "far outweighed" the 
costs, but two conditionally favored the 
benefits — one required that the reviewed unit 
needed to be committed to improvement and the 
other that the URC recommendations be 
implemented. Those who assessed that the costs 
outweighed the benefits provided these individual 
comments: the new chair was more influential 
than the review process; staff could have 
performed more research had they not been as 
involved in the review; the process is incredibly 
expensive, but no alternative was apparent; the 
funds involved could be better used; and an 
enormous amount of staff time was involved. 



Benefits. Respondents identified a variety 
of benefits, some quite general and others more 
specinc. The general benefits were identified in 
remarks such as the following: (a) the review 
was very useful and informative; (b) the impact 
has been profound and extremely useful; (c) the 
review led to organizational renewal and 
affirmation of thf, importance of the work of the 
unit; (d) the review made necessary and 
worthwhile contributions to the life and work of 
the University; (e) reviews have been a major 
success; (f) the reviews produced a very 
significant benefit to many, and probably most, 
of the departments reviewed; (g) a cathartic 
effect was produced and restorative action 
initiated; (h) a positive experience was provided 
for all involved; and (i) a positive attitude was 
developed towards evaluation. Four respondents 
stated that, by implementing a comprehensive 
review system, the University of Alberta was 
giving a clear message to government and others 
that it was serious about improving the quality of 
its operations. 

The more specific benefits related to these 
matters: (a) the impression that financial 
resource allocation had been placed on a better 
basis; (b) departments have been made more 
cohesive; (c) planning has been assisted; (d) an 
awareness of accountability, effectiveness, and 
purpose was generated; and (e) an enhanced 
communal appreciation of and responsibility to 
students, colleagues, the university, and society 
was developed. 

Costs. The direct annual central costs of the 
PACCR process were about $225,000 for 
PACCR operations (staff, PACCR office 
expenses, and URC expenses). Direct costs to 
units being reviewed mainly consisted of 
expenses involved in preparation of Self-Study 
Reports such as duplicating, holding of retreats, 
and hiring temporary staff. PACCR provided up 
to $500 per unit to assist with some of the direct 
costs. As noted by some respondents, the 
indirect costs to the units were probably more 
significant; several reported that substantial staff 
time was involved, leading to loss of 
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productivity in research and publication. 
Preparation of the Self-Study Report was 
frequently very demanding for the head of the 
unit. 

Self-study reports. These reports were 
generally well prepared, and many respondents 
considered that they constituted the most 
valuable part of the whole review process. 
Having a department involved in self-study 
activity was perceived to lead to more cohesion, 
to reflection by staff members about their place 
in their disciplines and the university, to focusing 
attention upon fundamentally important 
questions, to generation of greater awareness 
among staff of potentials and problems, to 
preparation of department plans, and to 
identification of aspects which needed 
strengthening and changes which were required 
in administrative practices in the department. 
Two respondents also commented on the benefit 
provided to senior administrators and to members 
of the Planning and Priorities Committee by 
reading the Self-Study Reports — they were able 
to learn a great deal about individual academic 
and support units. 

Unit Review Committees. Substantial 
support was evident for the review process to 
involve external non-partisan experts — 
"essential," "very important," "valuable," and 
"useful" were the terms commonly used. 
External reviewers were seen as "credible 
sources" of comment who could serve to validate 
conclusions of the Self-Study Report. These 
individual comments were provided: (a) 
comparison with other units could lead to 
promotion of excellence; (b) major benefits can 
be obtained if the URC is chaired by an 
excellent outside reviewer; (c) a major 
psychological boost can result if the image of the 
unit as excellent is confirmed; (d) the review is 
more useful if the external experts are willing to 
listen to ihe views of unit members; and (e) the 
review can enhance the image of the unit witii 
external experts. Use of both external and 
internal members on the URC was generally 
supported. 



PACCR. University administrators agreed 
that a small, knowledgeable group of senior 
respected staff members, not involved in day-to- 
day administration, formed an appropriate 
committee to organize the review process. Much 
of the success of the reviews was attributed to 
this committee. PACCR members were also 
praised for their ability to identify and elaborate 
upon important campus-wide issues. 

Follow-up activities. The extent to which 
foliow-up activities were conducted varied 
considerably among units. Several respondents 
identified such activities as being the weakest 
aspect of the review process. At the conclusion 
of the review activities, the President decided 
whether additional resources would be allocated. 
After that, successful follow-up was seen to 
depend largely upon efforts of deans and chairs. 
While respondents recognized that not all of the 
URC recommendations should necessarily be 
implemented, several remarked that more 
publicity should be given to both the President's 
decision and the activities of the faculty and 
department pursuant to the reconunendations. 

Involvement of the President. Active 
involvement of the President was universally 
viewed as a major factor which led to success of 
the PACCR reviews. The President was seen to 
perform important psychological, informational, 
and educational functions. However, some 
respondents considered that the President's active 
involvement in any future reviews should be 
somewhat reduced. 

Overall impressions. Two major attributes 
of the PACCR process were that it was 
perceived to be non-threatening and that 
academics came to recognize that reviews are 
essential for the health of the university. Over 
80% of respondents considered that the review 
process should continue and that many of the 
current practices should be retained. The 
greatest frustration experienced by those involved 
related to the lack of resources available to 
implement the URCs' recommendations. 
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Question 2: "Should some form of systematic 
reviews occur in the 1990s?" 

The distribution of responses was as follows: 
Yes-39; No-9; Undecided-2. Respondents also 
provided some elaboration either to justify or to 
elaborate upon their choice. Of the 39 who 
responded favorably, 17 specified these preferred 
time frames: once every 10 years (5); once every 
10-15 years (2); once every 8-10 years (2); after 
1995 (2); a different type for 1990-99, followed 
by PACCR type again in 2000 (1); more 
frequent than 10 years, but "stripped down*' (1); 
about every 7 years (1); after a 6-year gap (1); 
once every 20 or 30 years (1); and hold reviews 
of support units every 10 years and academic 
units every 15 years (1). Some of these 17 
respondents elaborated on their choice in these 
ways: (a) reviews provide an opportunity for 
redirection and stimulation; (b) the momentum 
should be continued; (c) the university 
conununity has the right to know of the benefits 
from the 1980-1990 reviews; (d) a broader base 
perspective [unspecified] is needed; (e) we 
should build on the 1980-1990 experience and 
omit irrelevant data; (f) retirements, demographic 
changes, and increasing percentages of adult 
students warrant continuation of the reviews; and 
(g) continuation is defensible if reasonable 
recommendations have a reasonable chance of 
realization. 

Those who were against continuation 
provided those comments: (a) once is sufficient; 
(b) a new vision is needed; (c) we should wait 
for five years before deciding; (d) cooperation 
would be reduced if PACCR continued; (e) 
reviews would meet hostility when doUars are 
short and workload is high; and (f) repetition 
would indicate that the 1980-90 experience was 
unsuccessful. The two who were undecided 
stated that (a) we need to obtain information 
from units about improvements which can clearly 
be linked to the PACCR process, and (b) while 
the reviews of supi>ort units are very worthwhUe, 
those of academic units are of questionable 
value. 



Question 3: "If your answer is yes to the 
previous question, then how would you 
structure the reviews for the 1990s?" 

Many suggestions were made about how any 
next cycle of reviews should be conducted: use 
the current PACCR model (7 respondents); use 
the current PACCR model but have more 
potential for budget adjustment (2); use the 
current PACCR model with the 1980s reviews as 
benchmarks (1); use the current PACCR model 
if the post-review process has worked well (1); 
and monitor progress now and start PACCR- 
style reviews again in 2000 (1). Four suggested 
that the procedures could be more streamlined, 
whJle one called for less rigidity. Five addressed 
the related matters of better follow-up activities, 
more feedback, greater capacity to fund 
recommendations, difficulties experienced in 
implementation, and ensuring that 
recommendations are implemented — these would 
aU require changes in procedures. 

With respect to the Self-Study Reports, two 
recommended tighter guidelines, two that the 
existing reports be used again, two that the 
existing reports be updated, one that annual 
reports be used for updating the rep/orts, and one 
that Self-Study Reports should be distributed to 
other departments in the same faculty. 

Only a few recommendations were made 
about the Unit Review Committees: use them 
only when necessary (1 respondent); use URCs 
consisting of three external experts (including at 
least one from outside Canada) and one 
University of Alberta staff member from a 
related field (1); the URCs should spend one 
week on campus during which the written report 
would be prepared (1); and a new-style URC — 
consisting of one external expert, one member of 
the original URC, and one University of Alberta 
member who would serve on several such 
committees — should be used to review a 
department's assessment of its successes and 
failures following the 1980s review (1). 

Several respondents recommended these foci 
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in any subsequent cycle: faculties as 
administrative units (5 mentions); appropriate 
groups of departments (3); units that require in- 
depth reviews again (3); examine the 1980s 
reviews and recommendations (3); undergraduate 
programs (2); our role in society (1); enhancing 
teaching effectiveness (1); new research thrusts 
(1); graduate programs (1); external research 
funding (1); staffing (1); capital equipment (1); 
committees (1); and research institutes (1). 

Negative Reactions 

In addition to the perceived costs and 
implied criticis^^is in the responses listed above, 
a few respondents identified these negative 
aspects of the first cycle: (a) recommendations 
of one URC were not useful and were discarded; 
(b) the process was divisive, as the URC was 
seen as either a savior or a threat; (c) no format 
was established within which the URC 
recommendations were to be implemented; (d) 
the reviews were rather superficial; (e) the 
reviewers rarely put the unit members 
sufficiently off balance in order to get at the 
truth; and (f) the departments were incorrectly 
called to account for developments and 
conditions over which they had no control. 

Recommendations 



The Examining Committee made the 
following recommendations: 

1. A second cycle of reviews be 
implemented at theUniversity of Alberta 
to evaluate units' proc;rams^ goals and 
priorities in terms of^ the mstitutional 
goals and objectives. 

2. The second cycle of reviews provide 
an information oase for th^^ effective 
allocation of human anc physical 
resources. 

3. The present PACCR process 
continue until all units have been 
reviewed, presumably by 1991, and that 
the second cycle begin without 
interruption. 

4. The senior administration consider 
the stated goals and objectives of the 
University so that a second cycle of 
reviews can be developed in harmony 



with the overall strategic plan of the 
University. 

5. The reviews in the second cycle 
start with the terms of reference' for 
each unit being set by a committee 
consisting of the Dean of the Faculty* 
the Chairperson of the Department, and 
a person from the senior administration 
who is intimately involved m plannmg. 
Full use should oe made of the reports 
dealing with the unit's first cycle of 
reviews. 

6. A Department Planning Report (no 
larger than 10 to 20 pages; be prepared 
by the unit before a site visit is 
initiated. This document should be 
accompanied by a complete set of 
curriculum vitae of all members of the 
academic staff in the unit^ and for 
administrative and service units the staff 

Profiles. [It would replace the Self- 
tudy Report used in the first cycle of 
reviews.] 

7. The entire cycle of reviews be 
completed within a five-year period, 
possibly followed by a hiatus before yet 
a third cycle is contemplated. 

8. The process continue to have the 
involvement of the President and/oi the 
Vice-Presidents. 

9. The reviews proceed in such a 
fashion that subject-related units be 
reviewed at the same time, or as closely 
as possible, so that redundancies may 
be reduced and beneficial interactions 
pursued. 

10. The PACCR committee remain a 
President* s committee, small in number 
(possibly four), and that the chairperson 
DC a member of the senior 
administration. The others should be 
respected personnel havmg extensive 
experience within the University, and 
refease time should be considered. 

11. The composition of the Unit 
Review Committees remain essentially 
unchanged. 

12. Greater emphasis be given by the 
senior administration to the evaluation 
and the follow-up of recommendations. 



Result 



Althougii the Examining Committee 
recommended a second review cycle in which 
some of the procedures used in the first cycle 
would be modified, the University decided not to 
undertake such an activity at this time. The 
main reasons for this decision related to current 
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fiscal problems and to the involvement of the 
University in an extensive strategic planning 
exercise. 

CONCLUDING COMMENTS 

Based upon the very positive experience of 
the University of Alberta with program reviews 
from 1980 to 1991, as described above, and the 
opinions of various authors, the following 
general recommendations are offered for 
postsecondary institutions contemplating 
implementation of a system of program reviews. 

1. Develop procedures slowly and 
thoughtfully, recognizing the need to generate 
trust and goodwill. Prepare an operating 
agreement and appropriate mechanisms which 
identify the purposes and the resources available, 
and emphasize the competence of reviewers. 

2. Review all acadeinic and support units, 
and review faculties and large support offices 
(e.g., physical plant) as overall administrative 
units after reviews of their constituent 
departments have been completed. 

3. Place primary emphasis upon program 
improvement, but also recognize that information 
from program reviews can help with budget and 
planning decisions. 

4. Use a small President's Advisory 
Committee chaired by an academic who is not 
currently an administrator, with all members 
being highly respected and sensitive members of 
the campus community. 

5. Incorporate the President's Advisory 
Committee into the institution's administrative 
structure and endeavor to ensure continuity of 
membership. 

6. Ensure that adequate and appropriate data 
are available throughout the review process. 



7. Use four reports in the review process: 
a Self-Study Report, a Unit Review Committee 
Report, a Unit Response, and a confidential 
President's Advisory Committee summary. 
Emphasize that the self-study activity will 
frequently be the most important phase of the 
process. 

8. Use one review team consisting of 
external experts and knowledgeable internal 
members. The chair should be one of the 
external members. 

9. Whenever possible and appropriate, 
attempt to integrate aspects of external 
accreditation and internal review processes to 
reduce the evaluative load upon professional 
departments. 

10. Ensure that the reviewing procedures 
are open and clearly understood, but treat the 
reports with appropriate confidentiality. 

11. When it is clear that poor administrative 
performance contributes to difficulties within the 
unit being reviewed, and even though the 
primary emphasis should be on program 
improvement, then senior administrators should 
be prepared to take appropriate firm action. 

12. Set aside a reasonable amount of money 
each year in a Review Adjustment Fund so that 
some units can receive increased funding in 
response to the reviews. 

13. The President's Advisory Committee 
needs to recognize that the subject of teaching 
loads is a contentious one in which there are 
vested interests. Depending on the assumptions 
made, almost any department can claim a heavy 
teaching load compared with other departments. 
TTie Committee should discourage units from 
making statements about these matters, which are 
likely to be unproductive, in their Self-Study 
Reports. We strongly recommend that 
President's Advisory Committees, and faculty 
members in general, carefully read pages 41-47 
of PACCR's Tenth Annual Report 
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14. Consider use of a written Memorandum 
of Agreement at the conclusion of the review 
process in order to ensure adequate follow-up 
activities. Charge deans, associate vice- 
presidents, and directors, as appropriate, with 
responsibility for monitoring implementation of 
such Memoranda. Allocate responsibility for 
monitoring overall implementation of 
Memoranda of Agreement with the appropriate 
Vice-President. 

15. Request that the President provide 
public annual reports in which the allocations to 
units from the Review Adjustment Fund are 
described. 

16. Ensure that the reviews consider the 
teaching, research, and service activities of the 
units, as appropriate; that they obtain the 
opinions ( f students in an organized manner; and 
that in th )k emphasis upon improvement they 
resist atte npts to rank imits in comparison with 
peer units in other institutions. 

In conclusion, we wish to state that the 
review experience at the University of Alberta 
has been overwhelmingly positive and that we 
agree with Taylor et al. (1988) that all 
postsecondary institutions should be involved in 
some form of continuous review activity. Of 
course, not all of the procedures that we adopted 
and used were perfect and not all aspects of the 
review experience met with campus-wide 
approval. But the minor deficiencies and costs 
of any review system which has been carefully 
planned should not substantially detract from its 
overall benefits. Serious resource and planning 
difficulties are confronting most -postsecondary 
institutions; the information obtained from 
reviews can be of substantial significance when 
these difficulties are being addressed. Apart 
from that benefit, the potential gains in 
improvement of quality of many aspects of 
academic and support operations warrant 
adoption of some thorough, ongoing system of 
institutional self-examination. 
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